Field of the Invention
The present disclosure relates to the generation of test scenarios during test designing in software testing. More particularly, the present disclosure envisages creating structured, indexed and annotated use cases and generating test scenarios therefrom.
Description of the Background Art
Software testing is a verification technique utilized in the software industry. Prior to testing software applications, it is necessary to identify test scenarios that accurately represent the behavioral characteristics of the software application to be tested. A test scenario is a well-defined collection of test cases which dictates the order in which test cases need to be executed in order to determine if a software application is working appropriately.
Testing software applications is a critical step in the application development lifecycle and appropriate coverage of the critical test scenarios is of prime importance. Test scenarios are used for determining the coverage of an application's specified requirements. Test scenarios may be identified either manually (by experienced Test Designers) or in some cases, using tools. Commonly used approaches include trying to generate/identify the test scenarios from an activity diagram.
Nowadays, with the advent of technology, testing systems that automatically execute test cases without any sort of manual intervention are available, but the creation of test cases demands human involvement and supervision. Manual creation of test cases requires application of human intelligence and deep understanding of the software application to be tested. The process of manual identification of appropriate test cases is often considered to be complicated and time consuming.
Use case modeling is a process of creating relevant use cases that correspond to a particular software application. Use cases are well established means for requirements elicitation and specification and modeling the behavior of a system from user's point of view. A use case is a scenario that describes the use of a system by a user to accomplish a certain goal. The user acts on the system by providing inputs to the system. A use case scenario is an ordered set of actions and responses that describes the interaction between the system and user(s) external to the system. Therefore a use case could be termed as an ordered collection of all the actors acting upon the system and their respective interactions with the system. Use cases are used in the process of software testing because of the below mentioned advantages:
use cases help capture the system's functional requirements from a user's perspective;
use cases provide for the active involvement of users in the process of requirements gathering;
use cases make it possible to identify major classes contained in a software application and also help in understanding the interrelationships between those major classes;
use cases are used as a foundation to create test scenarios or test cases;
since use cases are widely used as the basis for software development, utilizing use cases for the purpose of software testing provides a uniform notation and also promotes reusability of use cases; and
since the same set of use cases are utilized for software development as well as software testing, test cases to test the system could be developed in parallel with the development of software application thereby enabling a shorter lifecycle;
While tests—identified by designing test cases (and creation of corresponding test scripts)—are to be executed to validate the software application's conformance to the use case's requirements, due to the typically large number of possible combinations of actions, input data values, etc. in a system, the number of tests that can be run are often limited based on practical considerations such as time or effort. For targeting optimal coverage of tests (that is, application flow paths) for an application, it is therefore important to identify the critical and important test scenarios.
One of the common approaches utilized to test software applications is use case model based testing approach. In case of use case model based testing approach a use case model accurately representing the actual characteristics and expected behaviors of the software application under test is created. Formal use case models created using the process of use case modeling are used as the basis to identify test scenarios. One example of such a model is the activity diagram, which is created using the well-known Unified Modeling Language (UML). Creation of activity diagram/model automatically from use cases is also possible, though, often this may be done manually.
Test scenarios are higher level abstraction of a set of functional steps that encapsulate a particular test. Test cases are a detailed variant of test scenarios and typically include test inputs, test data and expected test results. A typical test scenario typically includes one or more test cases. A test scenario therefore acts as a means to determine the scope of coverage corresponding to test cases. In case of business critical software applications, use cases are likely to be complex and an accurate determination of a critical set of test scenarios is deemed necessary for conducting efficient testing of the software application.
With advancements in modeling technology and with advancements available as far as the languages utilized for the purpose of creating models are concerned, attempts have been made to create systems that would analyze software requirement specifications/documents and automatically identify test related attributes including but not restricted to test criteria, test data, test case execution paths and test execution scripts, based on the structure of the requirement specifications/documents. With the availability of Unified Modeling Language (UML), the task of analyzing the structure associated with use case models has been simplified. Systems of the prior-art, which utilize the method of use case grammar based categorization, often do not succeed in identifying the relevant test scenarios because those systems use single pass lexical analysis. Moreover the systems that use single pass lexical analysis fail to perform many of the required complex optimizations thereby resulting in identifying test scenarios which do not provide an accurate representation of the concerned system's behavior.
Moreover, another disadvantage associated with using single pass lexical analysis systems to identify test scenarios is that those systems have no inbuilt structure to categorize the identified test scenarios and the lack of categorization of test scenarios is a major issue when numerous use cases are created in correspondence to a particular software application. The process of automatic identification of test scenarios based on the use case documentation is further complicated by the fact that use cases are documented using different formats such as Unified Modeling Language (UML) and Systems Modeling Language (SysML).
Some of the prior art systems for generating test cases are discussed below:
U.S. Pat. No. 7,930,682 granted to Siemens Corporation teaches a system and method for generating test frames in specification based testing by using a path coverage criterion. The method envisaged by the above mentioned US patent includes receiving a test design as input and constructing a context free grammar graph based on the test design. The method further includes analyzing the context free grammar graph to generate a first set of paths and emitting a set of test frames as output based on the first set of paths. The aforementioned patent envisages a system that accepts an ECFG (extended context free grammar) model based test design as an input to generate all possible test frames. The method advantageously reduces frequent back tracking with repeated random number generator overheads while systematically traversing a context free grammar graph of a test design.
U.S. Pat. No. 6,378,088 granted to John Thomas Mongan teaches a test generator that generates tests by randomly traversing description of the interface of a program being tested, thereby generating tests that contain randomly selected actions and randomly selected data. During the phase of execution, the tests generated in accordance with the above mentioned US patent manipulate the program being tested. The generated tests take the form of test scripts written in the language of a commercially available testing tool and include programming statements that invoke one or more functions of the application program being tested.
U.S. Pat. No. 6,671,874 granted to Sofia Passova teaches a method of developing a model of a computer program which represents the system requirement conditions and events as a set of positions and transitions, and maps the system requirements in a table as positions and optionally as transition designations. The method in accordance with the aforementioned US patent models the system requirements, design requirements and program code for a computer program.
U.S. Pat. No. 6,976,246 granted to Microsoft Corporation teaches a finite state model based testing system that has a user interface which enables a user to enter state information and transition information about a software application to be tested. The user interface further enables the user to initiate a model generation engine to generate a model of the software application based on the state information and transition information. The testing system further includes a graph traversal menu that enables the user to select a graph traversal program for the purpose of generating a test sequence of inputs for the software application from the state and transition information provided. The testing system further includes a test execution menu that enables the user to select a test driver program which reads the test sequence of inputs for the software application and executes the test sequence of inputs on the software application.
U.S. Pat. No. 6,701,514 granted to Accenture LLP envisages a system and method for test maintenance in an automated scripting framework. In accordance with this patent document, the system for test maintenance first develops plurality of test scripts that are then stored in centrally located database. A specific test script from the stored test scripts is then edited by a user, these user-edits of the specific test script are then propagated to the plurality of stored test scripts. The system as disclosed in this document uses database to store test scripts and test criteria. Based on the stored test criteria, relationship between the stored test script components is identified in order to develop test scenarios from the underlying relational database. Therefore, this system utilizes relational database functionality to combine test script component and develop test scenarios.
None of the Patent Documents mentioned above have addressed the issue of automatically generating test scenarios from indexed use cases which are unambiguous. Moreover, none of the above mentioned Patent Documents address the issue of processing use cases as inputs that are created using different formats such as UML and SysML. Therefore there is felt a need for a system that can appropriately index the steps in use cases and automatically identify relevant test scenarios based on the indexed use cases.